A trust region-type normal map-based semismooth Newton method for nonsmooth nonconvex composite optimization

We propose a novel trust region method for solving a class of nonsmooth and nonconvex composite-type optimization problems. The approach embeds inexact semismooth Newton steps for finding zeros of a normal map-based stationarity measure for the problem in a trust region framework. Based on a new merit function and acceptance mechanism, global convergence and transition to fast local q-superlinear convergence are established under standard conditions. In addition, we verify that the proposed trust region globalization is compatible with the Kurdyka-{\L}ojasiewicz (KL) inequality yielding finer convergence results. We further derive new normal map-based representations of the associated second-order optimality conditions that have direct connections to the local assumptions required for fast convergence. Finally, we study the behavior of our algorithm when the Hessian matrix of the smooth part of the objective function is approximated by BFGS updates. We successfully link the KL theory, properties of the BFGS approximations, and a Dennis-Mor{\'e}-type condition to show superlinear convergence of the quasi-Newton version of our method. Numerical experiments on sparse logistic regression and image compression illustrate the efficiency of the proposed algorithm.Comment: 56 page

Variational Properties of Decomposable Functions Part II: Strong Second-Order Theory

Local superlinear convergence of the semismooth Newton method usually requires the uniform invertibility of the generalized Jacobian matrix, e.g. BD-regularity or CD-regularity. For several types of nonlinear programming and composite-type optimization problems -- for which the generalized Jacobian of the stationary equation can be calculated explicitly -- this is characterized by the strong second-order sufficient condition. However, general characterizations are still not well understood. In this paper, we propose a strong second-order sufficient condition (SSOSC) for composite problems whose nonsmooth part has a generalized conic-quadratic second subderivative. We then discuss the relationship between the SSOSC and another second order-type condition that involves the generalized Jacobians of the normal map. In particular, these two conditions are equivalent under certain structural assumptions on the generalized Jacobian matrix of the proximity operator. Next, we verify these structural assumptions for $C^2$-strictly decomposable functions via analyzing their second-order variational properties under additional geometric assumptions on the support set of the decomposition pair. Finally, we show that the SSOSC is further equivalent to the strong metric regularity condition of the subdifferential, the normal map, and the natural residual. Counterexamples illustrate the necessity of our assumptions.Comment: 28 pages; preliminary draf

Nonmonotone globalization for Anderson acceleration via adaptive regularization

Anderson acceleration (AA) is a popular method for accelerating fixed-point iterations, but may suffer from instability and stagnation. We propose a globalization method for AA to improve stability and achieve unified global and local convergence. Unlike existing AA globalization approaches that rely on safeguarding operations and might hinder fast local convergence, we adopt a nonmonotone trust-region framework and introduce an adaptive quadratic regularization together with a tailored acceptance mechanism. We prove global convergence and show that our algorithm attains the same local convergence as AA under appropriate assumptions. The effectiveness of our method is demonstrated in several numerical experiments

Accelerating ADMM for efficient simulation and optimization

The alternating direction method of multipliers (ADMM) is a popular approach for solving optimization problems that are potentially non-smooth and with hard constraints. It has been applied to various computer graphics applications, including physical simulation, geometry processing, and image processing. However, ADMM can take a long time to converge to a solution of high accuracy. Moreover, many computer graphics tasks involve non-convex optimization, and there is often no convergence guarantee for ADMM on such problems since it was originally designed for convex optimization. In this paper, we propose a method to speed up ADMM using Anderson acceleration, an established technique for accelerating fixed-point iterations. We show that in the general case, ADMM is a fixed-point iteration of the second primal variable and the dual variable, and Anderson acceleration can be directly applied. Additionally, when the problem has a separable target function and satisfies certain conditions, ADMM becomes a fixed-point iteration of only one variable, which further reduces the computational overhead of Anderson acceleration. Moreover, we analyze a particular non-convex problem structure that is common in computer graphics, and prove the convergence of ADMM on such problems under mild assumptions. We apply our acceleration technique on a variety of optimization problems in computer graphics, with notable improvement on their convergence speed

Anderson acceleration for nonconvex ADMM based on Douglas-Rachford splitting

The alternating direction multiplier method (ADMM) is widely used in computer graphics for solving optimization problems that can be nonsmooth and nonconvex. It converges quickly to an approximate solution, but can take a long time to converge to a solution of high-accuracy. Previously, Anderson acceleration has been applied to ADMM, by treating it as a fixed-point iteration for the concatenation of the dual variables and a subset of the primal variables. In this paper, we note that the equivalence between ADMM and Douglas-Rachford splitting reveals that ADMM is in fact a fixed-point iteration in a lower-dimensional space. By applying Anderson acceleration to such lower-dimensional fixed-point iteration, we obtain a more effective approach for accelerating ADMM. We analyze the convergence of the proposed acceleration method on nonconvex problems, and verify its effectiveness on a variety of computer graphics problems including geometry processing and physical simulation

Variational Properties of Decomposable Functions. Part I: Strict Epi-Calculus and Applications

We provide systematic studies of the variational properties of decomposable functions which are compositions of an outer support function and an inner smooth mapping under certain constraint qualifications. We put a particular focus on the strict twice epi-differentiability and the associated strict second subderivative of such functions. Calculus rules for the (strict) second subderivative and twice epi-differentiability of decomposable functions are derived which allow us to link the (strict) second subderivative of decomposable mappings to the simpler outer support function. Applying the variational properties of the support function, we establish the equivalence between strict twice epi-differentiability of decomposable functions, continuous differentiability of its proximity operator, and the strict complementarity condition. This allows us to fully characterize the strict saddle point property of decomposable functions. In addition, we give a formula for the strict second subderivative for decomposable functions whose outer support set is sufficiently regular. This provides an alternative characterization of the strong metric regularity of the subdifferential of decomposable functions. Finally, we verify that these introduced regularity conditions are satisfied by many practical functions.Comment: 28 pages; preliminary draf

Descent Properties of an Anderson Accelerated Gradient Method With Restarting

Anderson Acceleration (AA) is a popular acceleration technique to enhance the convergence of fixed-point iterations. The analysis of AA approaches typically focuses on the convergence behavior of a corresponding fixed-point residual, while the behavior of the underlying objective function values along the accelerated iterates is currently not well understood. In this paper, we investigate local properties of AA with restarting applied to a basic gradient scheme in terms of function values. Specifically, we show that AA with restarting is a local descent method and that it can decrease the objective function faster than the gradient method. These new results theoretically support the good numerical performance of AA when heuristic descent conditions are used for globalization and they provide a novel perspective on the convergence analysis of AA that is more amenable to nonconvex optimization problems. Numerical experiments are conducted to illustrate our theoretical findings.Comment: 23 pages; 4 figure

The ‘Perfect’ Conversion: Dramatic Increase in CO2 Efflux from Shellfish Ponds and Mangrove Conversion in China

Aquaculture, particularly shellfish ponds, has expanded dramatically and become a major cause of mangrove deforestation and “blue carbon” loss in China. We present the first study to examine CO2 efflux from marine aquaculture/shellfish ponds and in relation to land-use change from mangrove forests in China. Light and dark sediment CO2 efflux from shellfish ponds averaged at 0.61 ± 0.07 and 0.90 ± 0.12 kg CO2 m−2 yr−1 (= 37.67 ± 4.89 and 56.0 ± 6.13 mmol m−2 d−1), respectively. The corresponding rates (−4.21 ± 4.54 and 41.01 ± 4.15 mmol m−2 d−1) from the adjacent mangrove forests that were devoid of aquaculture wastewater were lower, while those from the adjacent mangrove forests (3.48 ± 7.83 and 73.02 ± 5.76 mmol m−2 d−1)) receiving aquaculture wastewater markedly increased. These effluxes are significantly higher than those reported for mangrove forests to date, which is attributable to the high nutrient levels and the physical disturbance of the substrate associated with the aquaculture operation. A rise of 1 °C in the sediment temperature resulted in a 6.56% rise in CO2 released from the shellfish ponds. Combined with pond area data, the total CO2 released from shellfish ponds in 2019 was estimated to be ~12 times that in 1983. The total annual CO2 emission from shellfish ponds associated with mangrove conversion reached 2–5 Tg, offsetting the C storage by mangrove forests in China. These are significant environmental consequences rather than just a simple shift of land use. Around 30% higher CO2 emissions are expected from aquaculture ponds (including shellfish ponds) compared to shellfish ponds alone. Total annual CO2 emission from shellfish ponds will likely decrease to the level reported in early 1980 under the pond area-shrinking scenario, but it will be more than doubled under the business-as-usual scenario projected for 2050. This study highlights the necessity of curbing the expansion of aquaculture ponds in valuable coastal wetlands and increasing mangrove restoration to abandoned ponds

The ‘Perfect’ Conversion: Dramatic Increase in CO₂ Efflux from Shellfish Ponds and Mangrove Conversion in China

Aquaculture, particularly shellfish ponds, has expanded dramatically and become a major cause of mangrove deforestation and “blue carbon” loss in China. We present the first study to examine CO2 efflux from marine aquaculture/shellfish ponds and in relation to land-use change from mangrove forests in China. Light and dark sediment CO2 efflux from shellfish ponds averaged at 0.61 ± 0.07 and 0.90 ± 0.12 kg CO2 m−2 yr−1 (= 37.67 ± 4.89 and 56.0 ± 6.13 mmol m−2 d−1), respectively. The corresponding rates (−4.21 ± 4.54 and 41.01 ± 4.15 mmol m−2 d−1) from the adjacent mangrove forests that were devoid of aquaculture wastewater were lower, while those from the adjacent mangrove forests (3.48 ± 7.83 and 73.02 ± 5.76 mmol m−2 d−1)) receiving aquaculture wastewater markedly increased. These effluxes are significantly higher than those reported for mangrove forests to date, which is attributable to the high nutrient levels and the physical disturbance of the substrate associated with the aquaculture operation. A rise of 1 °C in the sediment temperature resulted in a 6.56% rise in CO2 released from the shellfish ponds. Combined with pond area data, the total CO2 released from shellfish ponds in 2019 was estimated to be ~12 times that in 1983. The total annual CO2 emission from shellfish ponds associated with mangrove conversion reached 2–5 Tg, offsetting the C storage by mangrove forests in China. These are significant environmental consequences rather than just a simple shift of land use. Around 30% higher CO2 emissions are expected from aquaculture ponds (including shellfish ponds) compared to shellfish ponds alone. Total annual CO2 emission from shellfish ponds will likely decrease to the level reported in early 1980 under the pond area-shrinking scenario, but it will be more than doubled under the business-as-usual scenario projected for 2050. This study highlights the necessity of curbing the expansion of aquaculture ponds in valuable coastal wetlands and increasing mangrove restoration to abandoned ponds